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In re Application of: 

Saint-Hilaire et al. 

Serial No. 09/989,136 

Filed: November 20, 2001 

For: METHOD AND ARCHITECTURE TO 
SUPPORT INTERACTION BETWEEN 
A HOST COMPUTER AND REMOTE 
DEVICES ___ 



Mail Stop Amendment 
Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 22313-1450 



Examiner: Knapp, Justin R. 
Art Unit: 2182 



DECLARATION PURSUANT TO 37 C.F.R.S1.131 



Sir: 

We, Ylian Saint-Hilaire and Jim W. Edwards, hereby declare that: 

1 . We are the inventors of the above-captioned patent application and the 
subject matter described and claimed therein. 

2. Intel Corporation of Santa Clara, California, is the assignee of the above- 
captioned patent application. 

3. We were employed by Intel Corporation at the time the above-captioned 
patent application was filed. 

4. Prior to November 9, 2001 , We (at least one of us for each independent 
claim) conceived of the invention according to each of independent claims 1,6, 16, 22, 
29, 31, 36, and 41 of the above-captioned patent application in this country, as 
evidenced by Exhibit A, B, and C. The date stamp, in unredacted form, shows that the 
concept identified in each of these documents document was written prior to November 
9, 2001. 
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5. Exhibits A, B, and C are copies of Intel invention disclosures (with 
redacted dates) that Ylian Saint-Hilaire and Jim Edwards prepared and submitted to an 
Intel patent review committee prior to November 9, 2001. The invention disclosures 
concern subject matter disclosed in the above-captioned patent application. 

6. Ylian Saint-Hilaire met with patent attorney R. Alan Burnett for an 
invention disclosure meeting in Hillsboro, Oregon prior to November 9, 2001 to discuss 
the subject-matter of the above-captioned patent application. 

7. We reduced the present invention to practice, with due diligence from a 
date prior to November 9, 2001 to November 20, 2000 (the filing date of the above- 
captioned patent application) in this country. 

We hereby declare that all statements made herein of our own knowledge are 
true and that all statements made on information and belief are believed to be true; and 
further that these statements were made with the knowledge that willful false 
statements and the like are punishable by fine or imprisonment, or both, under Section 
1001 of Title 18 of the United States Code, and that such willful false statements may 
jeopardize the validity of the above-captioned application or any patent issued thereon. 



Respectfully submitted, 
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Rev. 15, 8/00 

INTEL INVENTION DISCLOSURE 

, ATTORNEY-CLIENT PRIVILEGED COMMUNICATION 

?j j\a€tjui ^) /) <9 g> ^ located at http://legaUntel.com ^j.cic^<.iec< Date. 

date- mm 1 1 1 1 — ^^jTXc i ^pf\& /tkc/ CeL mmmm 

It is important to provide accurate and detailed information on this form. The information will be used to evaluate 
your invention for possible filing as a patent application. When completed and signed, please return this form to 
the Legal Department at JF3-147. You can submit electronically via e-mail to "Invention disclosure submission" 
if all of the information is electronic, including drawings and supervisor approval. If you have any questions, 
please call 264-0444. 

1. ^ifiventor: Saint-Hilaire : Ylian 



Last Name First Name Middle Initial 
Phone (5031264-2188 M/S: JF1-273 Fax# (5031 264-1805 



Citizenship: Canadian WWID: 10610963 Contractor YES _ NO_JC_ 

Inventor E-Mail Address: ylian.saint-hilaire@intel.com _^ — _ __ 

Home Address: 1316 NE Cariaby wav #173 — , 

City Hillsboro State OR Zip 97124 Country US 



•Corporate Level Group (e.g. iABG, NCG, CEG) IAG Division _TRL Subdivision _CEL_ 



Superyisor* Jim Edwards . WWID 10065024 Phone (503) 264-8464 M/S: , JF1-273 



Inventor: Edwards _ Jim _ . 



Last Name First Name Middle Initial 
Phone (503) 264-8464 M/S: JF1-273 Fax # (503)264-1805 _ 



Citizenship: _US WWID: 10065024 Contractor YES_ NO_X_ 

Inventor E-Mail Address: iim.Edwards@intel.com : ; 

Home Address: 9567 NW Arborview Dr — • . 

City Portland State OR Zip 97229 Country US 



♦Corporate Level Group (e.g. iABG, NCG, CEG) IAG Division TRL Subdivision _CEL. 



Supervisor* Mark Abel WWID 10059928 Phone (503) 264-8483 M/S: JF1-273 



*lf you are unsure of this information, please discuss with your manager. 
(PROVIDE SAME INFORMATION AS ABOVE FOR EACH ADDITIONAL INVENTOR) 

2. Title of Invention: Remote PC Hosted devices — -~ 

3. What technology/product/process (code name) does it relate to (be specific if you can): 

Home Devices, home automation. PC in the Home, home adapter, home network. : 

4 Include several key words to describe the technology area of the invention in addition to # 3 above: Networks, wireless. Extended 

ec_ . — _ — , — _ . _ — RECEIVED 

5. Stage of development (i.e. % complete, simulations done, test chips if any, etc.): Concept . 

6. (a) Has a description of your invention been, or will it shortly be, published outside Intel: ^K^r-J^f?^ POf**- 

PATENT DATABASE GROUP 

NO: X YES: If YES, was the manuscript submitted for pre-publicat||\|f^)[o\g|gQ^^ TEAR/I 

IDENTIFY THE PUBLICATION AND THE DATE PUBLISH ED: ; . . 



(b) Has your invention been used/sold or planned to be used/sold by Intel or others? 
NO: X YES: DATE WAS OR WILL BE SOLD: 
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V 

(c) Does this invention relate to technology that is or will be covered by a SIG (special interest group)/standard/ 

or specification? 

NO:_X YES: Name of SIG/Standard/Specification: . 

(d) If the invention is embodied in a semiconductor device, actual or anticipated date of tapeout? : 

(e) If the invention is software, actual or anticipated date of any beta tests outside Intel: _ 

7. Was the invention conceived or constructed in collaboration with anyone other than an Intel blue badge employee 
or in performance of a project involving entities other than Intel, e.g. government, other companies, universities 

or consortia? NO: X YES: Name of individual or entity: ....... . . 

8. Is this invention related to any other invention disclosure that you have recently submitted? If so, please give the title and 
inventors: No 



PLEASE READ AND FOLLOW THE DIRECTIONS ON 
HOW TO WRITE A DESCRIPTION OF YOUR INVENTION 



Please attach a description of the invention to this form and include the following information: 

1. Describe In detail what the components of the invention are and how the 
invention works. 




Picture Frame Extended PC 



This invention describes a method by which a low-cost home appliance (or any device) would be 
built with minimal software and hardware. This device would be connected on the home network 
(or any network) and be completely driven by the PC. 

Using this invention a picture frame device can be built with no storage and minimal memory 
(when compared to similar devices today). The computer in the home detects all such devices 
using UPnP (Universal Plug & Play) or another service discovery mechanism. Once the PC 
detects a device, it obtains (e.g. using UPnP) further information about the device and its 
capabilities. 

The PC interacts with a device using a set of services offered by that device. These services may 
include "Remote Display", "Remote Audio", "Remote Input", "Remote Video", "Remote Software 
Update" and other services. Each service is discovered using UPnP or its equivalent. UPnP also 
provides a specific description on how to use the service. 

Remote Display 

This service would provide a simple TCP connection port on which the PC would send a stream 
of "Display this picture at position X,Y" type commands. Each of these commands makes the 
device overwrite its frame buffer with a new picture at the given position. The UPnP information 
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oh this service would include the screen size, the color depth, the file formats supported and if 
* synchronization is supported. The computer uses this information to process pictures to be in 
the format, color depth, and size (to prevent overflowing device's frame buffer) required for the 
device. The computer will also do any scaling needed (if a picture must be re-sized for a given 
device). 

An optional synchronization service allows the computer to place time markers within the picture 
command stream. For example: if the computer must animate a small portion of the screen at 
15 frames per seconds, the computer will insert time markers to indicate when this picture must 
be displayed. This allows the device to synchronize with its refresh rate and also eliminates 
network jitter that would cause the animation to be choppy. The display service sends back 
small messages indicating when it failed to meet the specified time requirements. This allows 
the PC to adapt the content to the device and network conditions. 

Remote Audio 

The UPnP description of this service includes the number of channels (speakers), the rate at 
which the device can play, the quality (number of bits) and supported sound formats (PCM for 
example). When applications running on the PC want to play sounds and music, the PC is 
responsible for mixing and converting this sound to a format the remote audio device will 
support. 

In one implementation of this service, a TCP socket is opened to the remote audio device (port 
number given by the service's UPnP description). Both the device and PC TCP sockets can be 
optionally configured to buffer only a small amount of data (depending on sound quality, 
network latency and audio latency desired). The device simply reads PCM sound from the TCP 
socket and fills its sound buffer. If the device is unable to do so, it notifies the PC that there is a 
problem. The PC can adapt the sound quality to meet the device's performance or new network 
conditions. 

Remote Input 

The devices that can be built using this invention include many types of devices with input 
ranging from no input at all to pen-input, keyboard, mouse and buttons. UPnP is used to 
describe the existence of the input service. The information provided may include type of 
device, commands supported (buttons, positions (pen, mouse), clicks, wheel), etc 

Some devices such as infrared remote controls might require code running on the PC to 
interpret the timing codes received and convert them to button push commands. The UPnP 
service might also be used in this case to advertise the presence of a custom input device and 
the URL link where the matching PC software required to support the device is located on the 
Internet. The PC can then automatically get and run this software (possibly signed, trusted 
software). 

Other services such as Video, Software update, etc can be advertised and used in the same way. 
One of the most important points of this invention is that we are not defining a single "remoting 1 * 
protocol, but rather a set of protocols, one for each service offered by the device. Each protocol 
is also defined to adapt to the specifics of the device. For example, some devices will play 
different audio formats at different rates; use different screen sizes at different color depths. The 
computer discovers this and adapts to it. This is also an important difference since, for example, 
current products require 640x480 display, mouse input at a minimum. 



Technical Points 

• The main goal and benefit is to drive much of the work that might be performed by a 
device onto the computer. This will result in cheaper devices. 

• Each device may include one or more of the services described here. For example, some 
devices may not have an Input or a screen. This does not exclude them from using this 
invention. Each device advertises the services it has. 



Rev. 15, 8/00 
• 



The computer can drive many of these devices at the same time. Each device can be^ 
driven autonomously, but the computer can also create logical connections between fhe 
devices. For example a tablet device could be used as input to a picture frame device. 



2. Describe advantage(s) of your invention over what is done now. 

Currently, devices such as picture frames, tablets, setup boxes run fairly autonomously. With the 
existence of an always on, always connected Extended PC, these cheap devices can leverage 
the power of the PC to significantly lower their costs and make it possible to reuse existing 
capabilities provided by the PC which translates into lower TTM. 

3. YOU MUST include at least one figure illustrating the invention. 

If the invention relates to software, include a flowchart 
or pseudo-code representation of the algorithm. 




Home Network (Wired or Wireless) 




4. Value of your invention to Intel (how will it be used?). 

The Extended PC strategy attempts to "make the PC relevant in the home", this is in the face of a 
growing lineup of products that run in the home without the involvement of the PC. This invention 
adds value to devices through the PC's inherent processing, storage, and connectivity 
capabilities and it also makes it possible to build cheaper devices. This invention will be used in a 
reference design for a PC/CE adapter that links the PC to the entertainment system in a family 
room. 



5. Explain how your invention is novel. If the technology Itself is not new, 
explain what makes it different. 

Remote display techniques have been focused on optimizing bandwidth and on a pull model 
where clients connect to large servers that host applications. For example: remote administration 
of servers using dialup lines or the Internet, or remote online assistance thru the Internet. This 
invention is completely different in that the PC now hosts a portion of functionality the customer 
would otherwise have to buy within each device. It also uses a push model where the PC 
automatically discovers and pushes content to the devices. 

One of the most important differences with things that are currently done in this space is the 
device capability discovery. Contrary to other "terminal" products, this invention discovers the 
devices capabilities at runtime, in the invention; a monitor, user input, audio playback are all 
optional components of a device. The computer discovers what services (display, audio...) the 
device offers and loads software that will take advantage of these services. 



6. Identify the closest or most pertinent prior art that you are aware of. 

While many remoting products exist: Windows Terminal Server, WinFrame, X-Windows, PC 
Anywhere, Carbon Copy... They don't use the same technology and are in a separate field of 
use. There is no known prior art in this space. 
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7. Who is likely to want to use this Invention or infringe the patent 
if one is obtained and how would infringement be detected? 

Devices that rely on the PC to be turned on to work might be potentially infringing on this 
invention. Discovery protocols such as UPnP are easy to examine using a network packet sniffer. 
Protocols such as UPnP can easily be probed to look at the services they offer and information 
given about each service. Network sniffers can also look at how the protocols are used with each 
service. 



DATE: 



HAVE YOUR SUPERVISOR READ, DATE AND SIGN COMPLETED FORM 
OR FORWARD IT ELECTRONICALLY VIA E-MAIL TO INTENTION^DISCLpSURE_SUBMISSION 



SUPERVISOR: 







BY THIS SIGNING, I (SUPERVISOR) ACKNOWLEDGE THAT I HAVE READ AND UNDERSTAND THIS 
DISCLOSURE, AND RECOMMEND THAT THE HONORARIUM BE PAID 



Exhibit B 



F/sv. 15,8/00 

^ ^ INTEL INVENTION DISCLOSURE 

^/i^r^ZP ATTORNEY-CLIENT PRIVILEGED COMMUNICATION 
n ^^Cx^-^ / ^ located at http://legal.intel.com 

date: jmmm — _r: — _ -till / /^l/ ceu PxjatM 4afa 

It is important to provide accurate and detailed information on this form. The information will be used to evaluate 
your invention for possible filing as a patent application. When completed and signed, please return this form to 
the Legal Department at JF3-147. You can submit electronically via e-mail to "invention disclosure submission" 
if all of the information is electronic, including drawings and supervisor approval. If you have any questions, 
please call 264-0444. 

1. /tfWentor: Saint-Hilaire Ylian 

/ Last Name First Name Middle Initial 

Phone (503)264-2188 M/S: JF1-273 Fax# (503)264-1805 

Citizenship: Canadian WWID: 10610963 Contractor YES NO X 

Inventor E-Mail Address: Ylian.saint-hilaire@intel.com : — 

Home Address: 1 31 6 NE Carlabv way #1 73 . 

City Hillsboro State OR Zip 97124 Country US 

•Corporate Level Group (e.g. IABG, NCG, CEG) I AG Division TRL Subdivision CEL 

Supervisor* Jim Edwards WWID 10065024 Phone (503)264-8464 M/S: JF1-273 



*lf you are unsure of this information, please discuss with your manager. 
(PROVIDE SAME INFORMATION AS ABOVE FOR EACH ADDITIONAL INVENTOR) 

2. Title of Invention: Light Weight Remote Display & Synchronization 

3. What technology/product/process (code name) does it relate to (be specific if you can): 

Extended PC, Home Devices. PC in the Home, home adapter, home network. Display remotinq. 

4. Include several key words to describe the technology area of the invention in addition to # 3 above: Networks, wireless. Extended 

PC . . . . . . 

5. Stage of development (i.e. % complete, simulations done, test chips if any, etc.): Concept . ....... . . ' 



6. (a) Has a description of your invention been, or will it shortly be, published outside Intel: 

NO: X „ YES: If YES, was the manuscript submitted for pre-publication approval? 

IDENTIFY THE PUBLICATION AND THE DATE PUBLISHED: 



(b) Has your invention been used/sold or planned to be used/sold by Intel or others? 
NO: X YES: DATE WAS OR WILL BE SOLD: _____ 



fadoofed date. 
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(c) Does this invention relate to technology that is or will be covered by a SIG (special interest group)/standard/ 

'or specification? 

NO:_X YES: Name of SlG/^tandard/Specification: — __ 

(d) If the invention is embodied in a semiconductor device, actual or anticipated date of tapeout? _ , 

(e) If the invention is software, actual pr anticipated date of any beta tests outside lntel:_ ^ „ _ . 

7 Was the invention conceived or constructed in collaboration with anyone other than an Intel blue badge employee 
or in performance of a project involving entities other than Intel, e,g. government, other companies, universities 
or consortia? NO:_X. YES: „ Name of individual or entity: . ■ . ■ — 

8. Is this invention related to any other invention disclosure that you have recently submitted? If so, please give the title and 

inventors: No . . . _ — . : ■ : 



PLEASE READ AND FOLLOW THE DIRECTIONS ON 
HOW TO WRITE A DESCRIPTION OF YOUR INVENTION 



Please attach a description of the Invention to this form and include the following information: 

1 . Describe in detail what the components of the Invention are and how the 
invention works. 




Picture Frame Extended PC 

This invention describes a method for remoting a display to a low-cost networked home appliance 
(or any type of networked device with a display) that can be built with minimal software and 
hardware. This device can be connected anywhere on the home network (or any network) and be 
completely driven by the PC, Using this invention any PC application can display an interface on 
a remote device and synchronize small video and animations on the remote device. 

Examples of display devices include a picture frame, a network tablet, an info terminal for kitchen 
management tasks, and a television display adapter, all of which are appropriately connected to 
the home network through a wired or wireless connection (e.g. 802.11a, 802.1 1b, Ethernet, 
HomeRF, HomePlug, 1394, etc.). Each of these devices advertises by some means the presence 
of a display on the network. A PC detects the presence of a display and sets up a data 
connection to the display. 

Many of the devices as described above don't require large screen multimedia video but rather 
simple interaction with a user in order to accomplish specific tasks. For example, displaying 
pictures on a picture frame device only requires a simple user interface and simple 
images/animations (such as animated GIF's). In order to keep the device simple and as low cost 
as possible, the protocol used to talk to the device is kept very simple. In order to accomplish this 
goal, a basic set of primitives is supported. For example: 
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> \ Reset (int Protocol Version): 

o This primitive is used by the PC to reset the remote display. The display must clear itself 
and await instructions. This primitive also clears any state the device may have. The 
protocol is defined with minimal state, but reset is defined such as to future proof the 
device. If future protocol versions are defined, this command can be used to set which 
protocol version is going to be used by the PC, 

• Flush (int TimeSinceLastSync) 

o Some display devices may support double buffering, where the device writes images to a 
hidden buffer and only displays the hidden buffer to the screen when asked to do so. 
Double buffering results in less display flicker. The HlmeSinceLastSync? parameter 
specifies the number of clock cycles to wait before issuing the Flush command. 
For example, suppose an NTSC TV adapter has double buffer support and it keeps track 
of the screen refresh (about 30 times per second). The device draws into its hidden 
buffer. When it receives a Flush(O) command, it displays the hidden buffer to the screen 
at the next screen refresh. 

If the PC wants to animate a GIF at 15 frames per seconds, it will need to send a 
animation frame to the device followed by a Flush(2) command. This will instruct the 
device to wait 2 refresh cycles before executing the Flush command. 
If an animation frame is not received in time due to network and device loads, another 
mechanism comes into play to allow a PC to adaptively recover: If more than 2 refresh 
cycles occur before the reception of the Flush(2) command, a FlushFailedQ command is 
returned to the PC. This allows the PC to adapt the frame rate and animation size to 
better fit the current network load and ability of the device. 

• DrawFillBox (x1 , yl , x2, y2, Color) 

o Draw a fill box of the given color between the two point (x1 ,y2) and (x2,y2). 

• Drawlmage (x, y, image) 

o Draw an image at the (xt ,x2) location. Here, the image can be in many different formats. 
When the PC discovers the device on the network, it must get the screen size, color depth 
and supported image formats. Information such as image width, height, and color are all 
encoded within the image format. The PC may use special features of some image types 
such as alpha-blending and transparency provided the device supports it. 

• PlaceVideoBox (x1, y1, x2, y2) 

o In cases where the device supports streaming video as a separate service. This command 
allows the PC to place the video in a portion of the screen. Generally, the PC will set up 
the position of the video window and start streaming the video via a separate service. 

• Repaint ( ) 

o On occasion a device may lose the entire display content. To recover, the device can ask 
the PC to resend a complete update of the screen via the Repaint command. 

• FlushFailed ( ) 

o See Flush command. This command allows the PC to kept track of the quality of the 
animated output and adjust accordingly. 

Technical Points 

• Only ResetQ and DrawlmageQ are required commands. All other commands are optional. 
The list of supported commands, display size, color depth and other information is given to 
the PC at discovery time. The PC must adapt to the restrictions of the device. 

• Perfectly synchronized animations can be played on a device that supports the Flush() and 
FlushFailedO commands since the device animates the frames based on its own clock. If 
these commands are not available, the PC can still animate, but the quality of the animation 
is subject to network conditions. 

• As defined, the Flush command synchronizes based on a device side clock. It is also 
possible to refine this command to also synchronize based on audio time stamps. 

• Devices can also optionally support a Move() command that moves a block from a portion of 
the frame buffer to another. This allows for quick scrolling of graphics into the frame buffer. 

2. Describe advantage(s) of your invention over what is done now. 



Currently, devices such as picture frames, tablets, setup boxes run fairly autonomously. WitR the 
existence of an always-on, always-connected Extended PC, these devices can leverage the 
power of the Extended PC to significantly lower their costs and make it possible to reuse existing , 
capabilities provided by the PC (which translates into lower TTM). 

This invention provides an excellent, cost effective way of building a remote display of any kind. 
It is much lighter weight than similar "remote display" terminal software used on the PC. It also 
allows for synchronization of the frames to provide excellent visual quality for small animations 
and small video. 

3. YOU MUST include at least one figure Illustrating the invention. 
If the invention relates to software, include a flowchart 
or pseudo-code representation of the algorithm. 
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4. Value of your invention to Intel (how will it be used?). 

The Extended PC strategy attempts to "make the PC relevant in the home", in the face of a 
growing lineup of products that run in the home without the involvement of the PC. This invention 
adds value to devices through the PC's inherent processing, storage, and connectivity 
capabilities. It also makes it possible to build cheaper devices. This invention will be used in a 
reference design for a PC/CE adapter that links the PC to the entertainment system in a family 
room. 

5. Explain how your invention is novel. If the technology itself is not new, 

explain what makes it different. 

Remote display techniques have been focused on optimizing bandwidth and on a pull model 
where clients connect to large servers that host applications. For example: remote administration 
of servers using dialup lines or the Internet, or remote online assistance thru the Internet This 
invention is completely different in that the PC now hosts much of the functionality the customer 
would otherwise have to buy within each device. This remote display invention uses a push model 
where the PC constantly push's content to the devices. 



6. Identify the closest or most pertinent prior art that you are aware of. 

While many remoting products exist (e.g. Windows Terminal Server, WinFrame, X-Windows, PC 
Anywhere, Carbon Copy) they don't use the same approach to display remoting and are in a 
separate field of use. There is no known prior art in this space. 
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7. 



Who is likely to want to use this invention or infringe the patent 
if one is obtained and how would infringement be detected? 



Devices that rely on the PC to be turned on to work might be potentially inf ringing on this 
invention. Network sniffers can look at how the protocols are used with each service. 
Organizations that build low-cost devices that include a display with a PC-hosted user interface 
may infringe on the Invention, Analyzing the protocol used would detect the infringement. 



HAVE YOUR SUPERVISOR READ, DATE AND SIGN COMPLETED FORM 
OR FORWARD IT ELECTRONICALLY VIA E-MAIL TO INTENTION J)ISCLOSURE_SUBMISSION 



DATE; 




SUPERVISOR: 




BY THIS SIGNING, I (SUPERVISOR) ACKNOWLEDGE THAT I HAVE READ AND UNDERSTAND THIS 
DISCLO$URE, AND RECOMMEND THAT THE HONORARIUM BE PAID 
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o ir>->-n WTEL INVENTION DISCLOSURE 

(X / 00 I ATTORNEY-CLIENT PRIVILEGED COMMUNICATION 

' located at http://legal.intel.com 

DATE:jgL_ . T£<j /ft 1 */ T&V 

It is important to provide accurate and detailed information on this form. The information will be used to evaluate 
your invention for possible filing as a patent application. When completed and signed, please return this form to 
the Legal Department at JF3-147. You can submit electronically via e-mail to "invention disclosure submission" 
if all of the information is electronic, including drawings and supervisor approval- If you have any questions, 
please call 264-0444. 



1. Inventor: Saint-Hilaire . Ylian 



Last Name First Name Middle Initial 

Phone (503)264-2188 M/S: JF1-273 * Fax # (503) 264-1805 



Citizenship: Canadian _ WWID: 1Q610963 Contractor: YES NO_X. 

Inventor E-Mail Address: vlian.saint-hiiaire@inteLcx)m , ■ '. — 

Home Address: 1316 NE Cariabv wav #173 , : — 

City Hillsboro . . . . . . State OR Zip 97124 Country US 



-Corporate Level Group (e.g. iABG, NCG, CEG) IAG Division TRL Subdivision _CEL_ 



Supervisor* Jim Edwards WWID 10065024 Phone (503) 264-8464 M/S: JF1-273 



*if you are unsure of this information, please discuss with your manager. 
(PROVIDE SAME INFORMATION AS ABOVE FOR EACH ADDITIONAL INVENTOR) 

2. Title of invention: Adaptive Remote Input Service • . — 

3. What technology/product/process (code name) does it relate to (be specific if you can): 

Extended PC, Home Devices. PC in the Home, home adapter, home network. Display tempting. Input remoting ; — : ■ 

4 Include several key words to describe the technology area of the invention in addition to # 3 above: N9tvwfr3. wireless, Extended 
PC „ _ 

5. Stage of development (i.e. % complete, simulations done, test chips if any, etc.): Prototypes? • — 



6. (a) Has a description of your invention been, or will it shortly be, published outside Intel: 

NO: X _ YES: _ If YES, was the manuscript submitted for pre-publication approval? 

IDENTIFY THE PUBLICATION AND THE DATE PUBLISHED:. ■ ", : / : ; • " : 



(b) Has your invention been used/sold or planned to be used/sold by Intel or others? 
NO: X YES: DATE WAS OR WILL BE SOLD: 



RECEIVED 



patent Database group 

INTEL LEGAL TEAM 
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ATTORNEY-CLIENT PRIVILEGED COMMUNICATION 

(c) Does this invention relate to technology that is or will be covered by a SIG (special interest group)/standard/ 

or specification? 

NO: X YES: Name of SIG/Standard/Specification: . ■ — 
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7. Was the invention conceived or constructed in collaboration with anyone other than an Intel blue badge employee 
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8. Is this invention related to any other invention disclosure that you have recently submitted? If so, please give the title and 
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Please attach a description of the invention to this form and include the following information: 

1. Describe in detail what the components of the invention are and how the 
invention works. 




Television Extended PC 

Adapter 



This invention describes a method for remoting input from a low-cost networked home appliance 
(or any type of networked device with input) to a PC in such a way that the device can be built 
with minimal software and hardware. This device can be connected anywhere on the home 
network (or any network) and can be completely driven by the PC. Using this invention any 
device can send user input information to a PC that will interpret the user input and forward the 
command to the PC hosted device application. 

Examples of input devices that can take advantage of this invention include picture frames, 
network tablets, information terminals, kitchen management devices and television display 
adapters, all of which are appropriately connected to the home network through a wired or 
wireless connection (e.g. 802.11a, 802.11b, HPNA, Ethernet, HomeRF, HomePlug, 1394, etc.). 
Each of these devices advertises by some means (UPnP for example) the presence of a user 
input source on the network. The discovery system (UPnP) also tells the PC of the type of user 
input that can be expected from the input source: remote control, keyboard, mouse/pen/touch or 
custom. A PC detects the presence of an input source and sets up a data connection to it. It then 
receives the user input and dispatches the user events to an application. 

In some cases, the device will send custom input data, or in other words, raw data generated by 
the user input. This could include, for example, the length of IR pulses generated by a TV remote 
control. During the discovery process, the PC is informed of all of the custom input "channels" 



that the device has. For each of these channels, the device informs the PC of where it can obtain 
a code module that can interpret the raw data. Giving the PC one or more URL's to the code 
module located On the Internet generally does this. The code is also identifies with a unique 
identification number, this allows the user to load the appropriate code module for a different 
source (Disk, CPROM) and have the PC associate the code module to the correct input source. 

A basic set of primitives is supported by this lightweight remote input service. These primitives 
could include: 

• KeyDown (int keydata) 

• Keypress (int keydata) 

• KeyUp (int keydata) . 

o The three keyboard primitives match the well-known keyboard events used is almost all 
operating systems. KeyDown and Key Up occur whenever a key is pushed or released, 
this generally includes all keys including control keys such as shift, Ctrl, alt and function 
keys. The KeyPress event contains the resulting interpretation of the keystrokes such as 
small and large capitalization and keyboard repetitions. 

• MouseDown (int x, int y, int buttondata) 

• MouseMove (int x, int y, int buttondata) 

• MbuseUp (int x, int y, int buttondata) 

o The three pointer primitives match the well-known keyboard events used is almost all 
operating systems. These primitives are not limited to mouse movements, any other 
pointing device such as trackballs, movement keys, pen input, touch sensitive screen and 
magnetic screens can also use them. During input service discovery, the PC is informed 
of the maximum values of x and y. If the device has a display, these pointer primitives 
may have a different resolution any may not match the x and y of the display. The PC 
could scale the pointer position to the display. 

• RemoteControlKey (int buttondata) 

o The framework also use a remote control primitive to send to applications button 
information that is not generally associated with a computer keyboard. These buttons 
include: Play, Pause, Forward, Next Track, VCR, DVD... Some frameworks could opt to 
not use the primitive and map these remote control keys to standard computer keyboard 
events. 

• Custom (int channel_number, byte[] custom_data) 

o All keyboard, mouse and possibly remote control primitives are dispatched to the 

applications as-is. For some devices, data processing must be done before a keyboard or 
mouse event can be generated. In order to lower the cost of the device, the device uses 
the custom primitive to send raw input data to the PC for interpretation. The PC than 
sends the raw data into the appropriate code module for interpretation, the code module 
will generally return mouse or keyboard primitives that can than by dispatched to the 
applications. Each channel number can contain completely different raw data that must be 
sent to a different code module for interpretation. 

Technical Points 

• A device does not have to use all of these primitives, it can use all or a subset of them. 
Some devices may opt to use only custom data as input. 

• An XML file will describe how the input service behaves, what primates are used, that are 
the bounds of mouse coordinates, that custom channels exist and where to obtain the code 
modules to interpret the raw input data. 

• The XML file is written by the manufacturer of the device and validated using a XML-Schema 
file (XSD file), the Extended PC can also use the XSD file to validate the devices XML file 
before it is parsed. 

• The code modules that is loaded by the Extended PC to support one or more custom 
channel should be signed and authenticated if it is loaded from the Internet, just like any 
other code modules downloaded from the Internet. 

• Custom input code modules should also be sand-boxed, in other words, prevented from 
tampering system the files and other software on the Extended PC. This provides an extra 
level of security. 
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• One more instantiation of this service would do a similar operation in reverse, A code 
module on the PC could be loaded to generate a custom command to a device. For 
example, if a device has an IR transmitter, the PC could load a code module that sends a an 
array of IR pulse timings to the device to re-transmission. 

2. Describe advantage(s) of your invention over what is done now. 

Currently, devices such as picture frames, tablets, setup boxes run fairly autonomously. With the 
existence of ah always-on, always-connected Extended PC, these devices can leverage the 
power of the Extended PC to significantly lower their costs and make it possible to reuse existing 
capabilities provided by the PC (which translates into lower TTM). 

This invention provides an excellent, cost effective way of building remote devices of any kind 
and remote the input of such devices to the PC. It istwilt to take maximum advantage of the 
PC's flexibility and processing power while diminishing the cost of building the device. 

3. YOU MUST include at least one figure illustrating the invention. 

If the invention relates to software, include a flowchart 
or pseudo-code representation of the algorithm. 




4. Value of your invention to Intel (how will it be used?). 

The Extended PC strategy attempts to "make the PC relevant in the home", in the face of a 
growing lineup of products that run in the home without the involvement of the PC. This invention 
adds value to devices through the PC's inherent processing, storage, and connectivity 
capabilities. It also makes it possible to build cheaper devices. This invention will be used in a 
reference design for a PC/CE adapter that links the PC to the entertainment system in a family 
room. 



5. Explain how your invention is novel If the technology itself is riot new, 
explain what makes it different. 

Most electronic devices in the home today are standalone, or connect to the PC using protocols 
that require the device to process user input, interpret the user's commands end act on them with 
the assistance of the PC. The invention is novel in that is pushes ev<eh more processing that 
would be otherwise done on the device to the PC and allows for lower cost devices. 
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- 6. Identify the closest or most pertinent prior art that you are aware of. 

While many remoting products exist (e.g. Windows Terminal Server, WinFrame, X-Windows, PC 
Anywhere, Carbon Copy) they don't use the same approach convey user input to the user. Many 
solutions are targeted to WAN applications and fairly rich remote clients. There is no known prior 
art in this space. 

7. Who is likely to want to use this invention or infringe the patent 
if one is obtained and how would infringement be detected? 

Devices that rely on the PC to be turned on to host applications on behalf of a device might 
infringe on this invention. Network sniffers can look at how the protocols handle user input. 
Organizations that build low-cost devices that include, remoting user input to a PC-hosted user 
interface may infringe on the Invention. Analyzing the protocol used would detect the 
infringement. 



HAVE YOUR SUPERVISOR READ, DATE AND SIGN COMPLETED FORM 
OR FORWARD IT ELECTRONICALLY VIA E-MAIL TO INTENTION_DISCLOSURE_SUBMISSION 



DATE: . SUPERVISOR: 



BY THIS SIGNING, I (SUPERVISOR) ACKNOWLEDGE THAT I HAVE READ AND UNDERSTAND THIS 
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